Papers with gradient update process
Understanding Attention for Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on whether local attention weights reflect the importance of input representations. |
| Approach: | They propose to analyze for each word token the following two quantities: its polarity score and its attention score, where the latter is a global assessment on the token’s significance. |
| Outcome: | The proposed model can be improved under conditions where the interplay between the two quantities can contribute towards model performance. |
Unlocking Continual Learning Abilities in Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learning models (LMs) incorporate old task data or task-wise inductive bias into LMs, but old data and accurate task information are often unavailable or costly to collect. |
| Approach: | They propose a rehearsal-free method that updates model parameters with large magnitudes . they found that the L1-normalized magnitude distribution is different when different task data is used . |
| Outcome: | The proposed method improves accuracy and performance on four CL benchmarks. |
RTFE: A Recursive Temporal Fact Embedding Framework for Temporal Knowledge Graph Completion (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for static knowledge graph embedding (SKGE) ignore the continuity of states of TKGs in time evolution. |
| Approach: | They propose a Recursive Temporal Fact Embedding framework to transplant SKGE models to TKGs and enhance the performance of existing TKGE models. |
| Outcome: | The proposed framework can be used to transplant SKGE models to TKGs and improve existing models for TKG completion. |